Skip to content

Update specification for directives for sys.implementation and sys.platform checks. - #2173

Open
Josverl wants to merge 1 commit into
python:mainfrom
Josverl:specs/sys.implementation.name
Open

Josverl wants to merge 1 commit into
python:mainfrom
Josverl:specs/sys.implementation.name

Conversation

@Josverl

@Josverl Josverl commented Feb 12, 2026 •

Copy link
Copy Markdown

This PR expands and clarifies the specification for version, platform, and Python implementation checks:

  • Adds checks for sys.implementation.name and sys.implementation.version.
  • Specifies supported comparisons, including in / not in with literal tuples, sets, and lists, and sys.platform.startswith().
  • Covers combining checks with and, or, and not.
  • Distinguishes required forms from optional extensions.
  • Adds examples and guidance on target configuration and defaults.

References :

@python-cla-bot

python-cla-bot Bot commented Feb 12, 2026 •

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
@srittau srittau added the topic: typing spec For improving the typing spec label Feb 20, 2026
@Josverl
Josverl force-pushed the specs/sys.implementation.name branch 2 times, most recently from 4151e84 to d7fbb9e Compare February 21, 2026 17:09
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
@Josverl
Josverl force-pushed the specs/sys.implementation.name branch from d7fbb9e to 62d90dc Compare February 22, 2026 12:56
@Josverl

Josverl commented Feb 22, 2026 •

Copy link
Copy Markdown
Author

I have updated the text to better clarify the exact comparisons for each of the supported attributes,
excluded the use of named attributes,
and did some more word-smithing to reduce unneeded verbosity.
I have also added highlighting to the examples, as a raiding aide.

@Josverl
Josverl force-pushed the specs/sys.implementation.name branch from 62d90dc to ff338e3 Compare February 22, 2026 13:08
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
@Josverl

Josverl commented Mar 30, 2026 •

Copy link
Copy Markdown
Author

Dear TC,
Is there a decision on this PR yet, or have I missed a step in the process?

If so kindly let me know how to proceed.

@JelleZijlstra JelleZijlstra left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm generally supportive but I think this change needs some more editing, and I'd like support in at least some type checkers (at least an open PR) before we start mandating this pattern.

Comment thread docs/spec/directives.rst Outdated
@Josverl
Josverl force-pushed the specs/sys.implementation.name branch from c575c15 to c951b00 Compare April 3, 2026 19:21
@Josverl
Josverl requested review from carljm and erictraut April 3, 2026 19:23
@Josverl

Josverl commented Aug 9, 2026 •

Copy link
Copy Markdown
Author

Dear typing council,
I have been waiting patiently , but there seems to be no progress on this.
As far as I can see I have actioned - or responded - to all feedback,
Reviewers have not responded to the updates they requested. Eric has stepped down for the TC since - so that is understandable.

However now it appears that this PR is in a Catch-22 situation;
Waiting for One or more typecheckers to implement this - but for typecheckers to implement this it should be in the spec.

Lacking response I plan close this in a few weeks

Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
@davidhalter

davidhalter commented Aug 9, 2026 •

Copy link
Copy Markdown
Collaborator

Waiting for One or more typecheckers to implement this - but for typecheckers to implement this it should be in the spec.

This is not the case at all. It is very typical for type checkers to implement features even if they are not in the spec. It is very helpful to have feedback from type checker authors to be able to understand potential problems with spec changes.

I think this has worked quite well in the past.and if a feature is useful enough there is usually a type checker that will implement it.

@carljm carljm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your patience, @Josverl. I'm not sure we've fully reached consensus yet on what should be supported here, but I also generally support making this clearly specified. I also think this still needs some editing to give a clear and unambiguous presentation of the requirements.

Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
@Josverl

Josverl commented Aug 10, 2026

Copy link
Copy Markdown
Author

Thanks all for the renewed attention and comments.

  • I'll try to clarify where I can do
  • and keep a list of topics that need a decision one way or another

@Josverl
Josverl force-pushed the specs/sys.implementation.name branch from b3b76c1 to 98e3697 Compare August 15, 2026 20:18
@Josverl

Josverl commented Aug 15, 2026 •

Copy link
Copy Markdown
Author

Again thanks for the feedback and suggestions to improve the spec.

I processed the comments and suggestion - and commented or marked them as resolved to keep track.
Please feel free to Unresolve or add additional comments as needed.
I have extended the Configuration section to help clarify how a type checker could be configured including default values or approaches for defaults.

Open topics.

1. 2-Tuple or 3-Tuple version comparisons.
see comments

  • requirement is for 2-Tuple checks
  • type checkers are allowed to also process 3-Tuple checks

2. Should in / not in membership checks use tuple, set or list.
intent was tuples, was updated to set , now waiting for consensus

* ``sys.platform in <tuple/set/list of string literals>``
* ``sys.platform not in <tuple/set/list of string literals>``
* ``sys.implementation.name in <tuple/set/listof string literals>``
* ``sys.implementation.name not in <tuple/set/list of string literals>``

3. Scope
as raised by @AlexWaygood
Is the scope of this change too large (it has grown since that comment) and should there be a implementation timeline / staged implementation.
I have refrained from answering this - as I am not the one that can do the implementation.

I think the commits should be squashed when merging - I've kept them separate for now.

@carljm

carljm commented Aug 17, 2026

Copy link
Copy Markdown
Member

We could eliminate open-question (2) and at least partially address (3) by just considering containment checks as out of scope to specify for now. Considering they don't add any new capability, just allow writing more concise code, they aren't really necessary. We could leave this out of the spec / conformance suite for now and up to individual type checkers. Similarly maybe for startswith. Leaving it out doesn't prevent type checkers from supporting it. And including it in the spec / conformance suite doesn't guarantee type-checker support, either, so it's unclear how soon some of these difficult-to-support patterns will be usable in practice anyway, even if they are included here.

@jorenham

Copy link
Copy Markdown
Collaborator

We could eliminate open-question (2) and at least partially address (3) by just considering containment checks as out of scope to specify for now. Considering they don't add any new capability, just allow writing more concise code, they aren't really necessary.

+1

Similarly maybe for startswith.

Unlike containment checks, startswith can't always be expressed using equality checks. For example sys.platform on FreeBSD on Python<3.14 includes the version number (docs) , so it can be "freebsd7", "freebsd8", ...., "freebsd15". This list will grow each two (?) years too. So if you want to check if you're on FreeBSD, you'll have to spell out each previous version + a bunch of future versions. So at least until Python 3.13 is EOL, I think that we should keep startswith.

@Josverl

Josverl commented Aug 17, 2026 •

Copy link
Copy Markdown
Author

by just considering containment checks as out of scope to specify for now

That would be disappointing, especially as one of the first responses in May'25 in DPO included :

Supporting both in and not in operators is straightforward.

But reality is that there different opinions based on different considerations, and

.startwith() was added based on review comments. More than happy to remove that to simplify matters.

@Josverl Josverl closed this Aug 17, 2026
@Josverl

Josverl commented Aug 17, 2026 •

Copy link
Copy Markdown
Author

Not closed in anger - just a typo

@Josverl

Josverl commented Aug 28, 2026

Copy link
Copy Markdown
Author

Gentle reminder for additional feedback.

Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst
Comment thread docs/spec/directives.rst Outdated
@JelleZijlstra

Copy link
Copy Markdown
Member

Here's a summary of what type checkers already support (courtesy of Codex):

Proposed behavior mypy Pyright ty Pyrefly
sys.version_info >= (M, m) and < (M, m) Yes Yes Yes Yes
sys.platform == literal and != literal Yes Yes Yes Yes
sys.platform.startswith(literal) Yes No Yes Yes
sys.platform in tuple No No Yes Yes
sys.platform not in tuple No No Yes Yes
sys.implementation.name == / != No No No No
sys.implementation.name in / not in No No No No
sys.implementation.version >= / < No No No No
Combining recognized checks with not, and, and or Yes Yes Yes Yes
Configure target Python version Yes Yes Yes Yes
Configure target sys.platform Yes Partial Yes Yes
Configure implementation name or version No No No No

So nobody supports any sys.implementation checks. Support for slightly more advanced sys.platform checks is a bit varied. I had Codex look through past GitHub issues and it wasn't able to find much demand for in/not in checks, which makes me think that part is maybe not worth the trouble. On the other hand, ty and Pyrefly already support those (in Pyrefly's case, apparently as a result of some of the discussion around this PR), and it doesn't seem like an onerous feature to add.

hauntsaninja pushed a commit to python/mypy that referenced this pull request Sep 1, 2026
This is proposed for the spec in python/typing#2173. It is trivial to
support in
mypy and seems useful, so let's add it.
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
carljm added a commit to astral-sh/ruff that referenced this pull request Sep 21, 2026
Inline list and set membership tests now preserve the values of their
individual elements, allowing ty to determine when a condition is always
true or false. With the target platform set to Linux, both `sys.platform
in ["linux", "windows"]` and `sys.platform in {"linux", "windows"}`
select only the matching branch, so an assignment there is inferred as
`Literal[1]`.

Reuse the precise element extraction used for iteration and membership
narrowing without changing the inferred container type. Lists and sets
use the existing tuple membership equality evaluator, including its
identity-or-equality behavior. For sets, assume equality is an
equivalence relation and equal objects have equal hashes. Checking
reflexive equality across union alternatives also improves existing
tuple membership inference.

Context is that in python/typing#2173 people are
wanting to specify that `sys.platform in ["linux", "windows"]` should
work. This makes it work in ty so that we can be fine with any outcome
of that specification change.

## Test plan

Mdtests cover platform-dependent branch reachability; positive and
negative membership in inline lists and sets; inferred literal values
and unions; fixed-length unpacking; stored mutable containers; custom
equality and identity; comparison chains; and preservation of enum-set
narrowing.

## Ecosystem report

The comparison for head `90111f10da9359722abb903351b89aaae0f9bcba` adds
four `redundant-condition-strict` warnings. The paasta and SciPy
warnings correctly identify disjoint membership checks. The dd-trace-py
warning exposes existing limitations in modeling dynamic `setattr`; the
Scrapy warning exposes existing attribute narrowing across method calls
and `await`. Equivalent equality checks already exhibit both limitations
on the base revision. These existing limitations are accepted for this
change. See the [full ecosystem
report](#28676 (comment))
for minimized examples and exact comparison details.
@Josverl

Josverl commented Oct 7, 2026

Copy link
Copy Markdown
Author

Just to avoid unneeded waiting:
I think I have responded to all review comments; but have not marked them as resolved (as earlier I was to eager to close)
Please let me know if further clarifications or updates are requested, or that things are clear.

Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
@Josverl

Josverl commented Oct 19, 2026 •

Copy link
Copy Markdown
Author

@carljm,
Sorry for the delay in addressing your latest suggestions & comments.

CI also seems to have forgotten that I did sign the CLA with all my identities, re-signing is not possible either.
Not sure what I can do about that.

@carljm carljm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the CLA signing failure is because your latest commit (unlike the previous ones) is authored by Jos_Verlinde@hotmail.com, which GitHub can't associate with your GitHub account. If you can amend that commit with a recognized email address and force push, I suspect the CLA signing check will pass.

Barring my two nits below, I think this PR is ready for conformance tests. @jorenham @davidhalter @JelleZijlstra @rchen152 any objections to the spec updates here, before @Josverl writes conformance tests?

Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
@jorenham

Copy link
Copy Markdown
Collaborator

I'll try to take a look this evening

@jorenham jorenham left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, thanks. I left some comments for some minor issues / nits / suggestions.

Comment thread docs/spec/directives.rst
Required forms and optional extensions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Type checkers must support the forms specified above as required. Support for additional

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consistency nit

Suggested change
Type checkers must support the forms specified above as required. Support for additional
Type checkers should support the forms specified above as required. Support for additional

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think that "should" can be combined with "required" in this sentence.
That would degrade the meaning of the "required" statements in the prior paragraphs, which I think is incorrect.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO you should just drop the sentence. The spec is clear in the "above" cases where a type checker has to support it and where it's optional, so the sentence feels unnecessary.

Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst Outdated
Comment thread docs/spec/directives.rst
…atform checks.

Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
@Josverl
Josverl force-pushed the specs/sys.implementation.name branch from d2c4133 to e612515 Compare October 26, 2026 10:53
@Josverl

Josverl commented Oct 26, 2026

Copy link
Copy Markdown
Author

@carljm, @jorenham
I have addressed the last few review comments,
and have now I squashed the different edits and rebased on current master.

Also have tried to make a start on the conformance tests in #2375.
Lets continue the conversation there.

CLA signature appears to have been caused by some parts of the CLA check assuming that smtp e-mail addresses are case sensitive, but that was solved simply.

@carljm

carljm commented Oct 26, 2026

Copy link
Copy Markdown
Member

@Josverl Why do we need to do conformance tests in a separate PR? I would find it much easier to review, approve, and merge the spec changes and conformance tests together, in the same PR. Conformance tests may reveal changes that we need to make to the spec, and we need to ensure they are consistent with each other.

@Josverl

Josverl commented Oct 26, 2026 •

Copy link
Copy Markdown
Author

@Josverl Why do we need to do conformance tests in a separate PR? I would find it much easier to review, approve, and merge the spec changes and conformance tests together, in the same PR. Conformance tests may reveal changes that we need to make to the spec, and we need to ensure they are consistent with each other.

Why do we need to do conformance tests in a separate PR

The original plan I was set out on was much simpler, and I explicitly started out with: "This is my first post to the Python discuss, so please be gentle."

And was guided by the TC decision process that documents:

After at least a week, the Council will make a decision, which may be to accept the change, reject it, or modify it.

This is my first PR to "the big" Python, it is open more than 10 months with over 150 comments,

  • the scope has significantly increased beyond my original intent
  • it is also more complete and (much) better formulated.

to be very clear, I could not have done this without all your help, but it has taken over 17 months since first discussion. ( with time-gaps both from all involved)
For me this has long passed the simple stage, Instead it feels like I am lifting weight above my class, with little guidance on the process.

On the conformance tests I replied earlier :

Makes sense to me, but TBH I personally would need to study how to define these, And happy to do that either in this PR or as a follow-up , but perhaps agree on the 'rules' first

So what are the rules for creating conformance test , for functionality that is not currently supported ?
I have read the paragraph that explains that in the repo a few times. that explains how to add a single test for existing functionality.

Note that the TC process document does not mention conformance test.
To me, the most logical order would be :

  • discuss the spec, write the spec , approve the spec , update the tools, test the conformance.
    I do see your point in needing to change that , but

To extend beyond that, is perhaps apparent you. To me it is not.

Yes, I can and have set an AI to work, and revised the results to avoid the most apparent mistakes.
The tests seem OK , and a HTML page is produced.
But I have limited trust in the result , and that is why that PR is marked explicitly as Draft.

What is the process after that ?
If that is waiting for all tools to adopt and update the results per feature per tool then I am unable to estimate the wall-time.
If that means another X months then please allow me to be honest; to me that is a too much to ask for a first PR.

So that is why I split them , because I would not want to loose this part, and currently I am unable to commit to finish something that I cannot fully understand the implications of.

I would welcome a conversation on the challenges and opportunities involved in a more private setting.

@JelleZijlstra

Copy link
Copy Markdown
Member

@Josverl are you on Discord? I'd be happy to connect there and talk through issues in a more informal way. (I can't speak for Carl but he's also on Discord and might also be willing to chat there.)

@Josverl

Josverl commented Oct 27, 2026

Copy link
Copy Markdown
Author

are you on Discord

Yes, thanks for the offer.

I shared my details via E-mail.

@carljm

carljm commented Oct 27, 2026

Copy link
Copy Markdown
Member

Yes, happy to discuss in Discord!

To answer a few specific questions/comments:

  • Reducing scope of a change to the spec is almost never a problem, IMO; if you would prefer to reduce scope of this change closer to your original scope so as to make it feel more manageable, feel very free to do so! Though it seems maybe we've by now done the work and reached agreement on the current scope, so it no longer makes sense to reduce.
  • I haven't done a detailed review yet, but at first glance the approach you are taking on the conformance tests looks good to me!
  • Implementation in type checkers is not a pre-requisite to changing the conformance suite. Only review and approval of the spec and conformance suite changes. This will mark every type checker as out of conformance on the new test file for now, but that's fine.

@davidhalter davidhalter left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First of all, thank you @Josverl for staying on this for such a long time. This is one of the most discussed issues and it's easy to give up at some point. Also thank you to Carl for doing most of the review work.

The PR is in a very good shape IMO. I have a few questions that are directed to other people than @Josverl, and a small suggestion to drop a sentence, otherwise no work should be necessary.

I would definitely not reduce the scope for now. While I would be grateful for conformance tests here I'd rather merge it as is and do the conformance tests later if @Josverl decides to not work on this anymore, since I think it's in a really good state and a massive improvement from the previous version. Though if you @Josverl still have some energy left in the tank, conformance tests would of course be awesome. :)

Thanks all!

Comment thread docs/spec/directives.rst
^^^^^^^^^^^^^^

Type checkers may optionally support comparisons of ``os.name`` with string
literals.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this optional? Sorry I haven't followed the discussion for a while and I'm sure it has been discussed. It might make sense to make this mandatory as well.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it's currently optional in the the conformance suite:

if os.name == "bogus_os":
val10 = ""
else:
val11 = ""
this_raises = val10 # E?: `val10` is undefined, but support for `os.name` is optional
does_not_raise = val11 # E? should not error if `os.name` control flow is supported, but might be flagged as possibly undefined otherwise
if os.name != "bogus_os":
val12 = ""
else:
val13 = ""
does_not_raise = val12 # E?: should not error if `os.name` control flow is supported, but might be flagged as possibly undefined otherwise
this_raises = val13 # E?: `val13` is undefined, but support for `os.name` is optional

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! IMO we should probably make that mandatory then in a separate follow up PR.

Comment thread docs/spec/directives.rst
Required forms and optional extensions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Type checkers must support the forms specified above as required. Support for additional

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO you should just drop the sentence. The spec is clear in the "above" cases where a type checker has to support it and where it's optional, so the sentence feels unnecessary.

Comment thread docs/spec/directives.rst
The configuration options should allow users to specify the target values for these
symbols, so that type checkers can evaluate the version and platform checks
correctly. The exact mechanism and names for these configuration options are
implementation-specific and defined by each type checker.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more of a question to other type checker authors: Is this already implemented somewhere? I definitely like that sys.implementation.name is added, that seems like a very useful addition especially in stubs. The implementation version is probably also needed, but that asks the question: How should I infer the implementation version? Should I just default to a nullable sys.implementation.version if the implementation cpython is not used and the version is not provided?

IMO it would be nice if we added this to the conformance tests as well, but that might be a little more complicated since we would need a way to specify the implementation/version for a specific test file. So I don't think this is necessary for this PR, but I would definitely appreciate if someone of us (looking at myself as well) would do the work here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic: typing spec For improving the typing spec

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants